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Patent Application 
Attorney Do. No. 2705-125 

SERVER AND METHOD FOR TRANSMITTING STREAMING MEDIA 
TO CLIENT THROUGH A CONGESTED NETWORK 

BACKGROUND OF THE INVENTION 

1 . Field of the invention 

The present invention is related to the field of data transmission over 
networks, and more specifically to devices and methods for adjusting a transmission 
rate of streaming media depending on available bandwidth. 

2. Description of the related art 

Networks are increasingly used to transmit data. The data can be for various 
media, such as audio or video, and is usually organized in groups that are called 
frames. When such data is transmitted continuously, it is also called streaming media. 
The transmission can either be for live, or for canned media. Either way, the 
transmission has to be in real-time, or in delayed real-time. 

A fundamental requirement for streaming media is that it has to be played at a 
constant rate. Packet networks, however, may transmit data asynchronously. 
Therefore, for streaming media to be transmitted over packet networks, strict timing 
requirements have to be met. Furthermore, where there are two related but separate 
streams of media (such as audio and video), they have to be time-synchronized in 
order to achieve high-quality media play out. 

A network channel should be used that has a higher bandwidth than what is 
required by the streaming media. The network, however, is occasionally subject to 
congestion, which constricts the bandwidth of the channel. This causes play out of 
streaming media to exhibit the problem of network jitter. This means that what is 
played out might be momentarily discontinued. 

Referring to FIG. 1, the problem of network jitter has been addressed in the art 
by introducing a de-jitter buffer at the receiving end. A server or proxy 110 receives 
data from a source, and stores it in a transmit buffer 114. The proxy 1 1 0 then 
transmits the stored data through a network 150 to a cUent or receiver 160. Typically, 
the client 160 includes a fixed size de-jitter receive buffer 162. 
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The transmit buffer 1 14 transmits data at the regular rate. It does not transmit 
any faster, because it if did, it would overflow the buffer 162. The buffer 162 first fills 
up to its size with data, and then starts playing out at the same rate as it is being filled. 
Momentary constrictions in bandwidth affect only the fullness of the buffer, not the 
play out, therefore diminishing network jitter. 

The buffer 162 does not work well where there are high-bit rate media 
streams, which are transmitted through heavily congested networks. In these cases 
network jitter still occurs, notwithstanding the buffer 162. 

Referring to FIG. 2, the phenomenon of network jitter is now described in 
more detail. Various quantities are plotted against a time axis, which is shown as a 
line 210. 

A solid, piece-wise Unear graph 220 represents the time evolution of the 
available bandwidth of network 150. Notice that the graph 250 has a nominal value 
BI, but which is not always the case. Indeed, for a heavily congested network, the 
graph 250 has two main bandwidth constrictions, which are represented by "wells" A 
and B. These are instances where the available network bandwidth drops to values 
below BI. Well A is deeper than well B, but does not last as long. 

A dashed line 230 represents the instantaneous rate of receiving data through 
the network. In FIG. 2, the line 230 starts at a value fy, which is the steady state, 
assuming no prior interruptions in the transmission. The value fy is also the 
streaming media play out rate. That is how fast the data stream is being transmitted 
from the proxy 110. In addition, line 230 is always less than the instantaneous value 
of graph 220. For example, the Une 230 tracks the graph 220, as it decreases in well 
A and well B. Immediately after the wells, however, the dashed line 230 increases, as 
formerly bottlenecked data is released from the network. After a long, uninterrupted 
regular receiving time, line 230 settles back at a value fy. 

A hne 240 represents the instantaneous fullness of the buffer 162. In FIG. 2, 
initially the buffer 162 is at its nominal size fy, after a long time of network 
congestion. 

The fullness of receiving buffer 162 remains at approximately the same level, 
until the first dip occurs (well A). Line 240 follows with a corresponding well A', 
and then recovers. In more mathematical terms, line 240 is determined from a time 
integral of the difference between line 230 minus the value fy. For example, if well A 
did not reach below fy, line 240 would not exhibit well A' at all. 
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Similarly, due to well B, line 240 exhibits a well B', and then recovers. Well 
B', however, reaches zero, with minor blips 246. This corresponds to a situation of a 
buffer under flow. Data is still arriving (line 230 is above zero), but slower than it is 
expected to be played (line 230 is less than value fy). Between time points 242, 244, 
play out is interrupted. This interruption is called network jitter. 

As will be appreciated, network jitter is a disruption in play out of the 
streaming media. If it is video, then the image stops, or flashes intermittently. If it is 
audio, then the sound is interrupted. In both cases, network jitter is annoying to the 
end user. 

BRIEF SUMMARY OF THE INVENTION 

The present invention overcomes the problems and limitations of the prior art. 

Generally, the present invention provides a system and a method that prevents 
network jitter, even in networks with seriously interrupted bandwidth. 

The invention senses the available bandwidth of the network. When the 
bandwidth is decreased, the invention transcodes the streaming media. This means 
that each frame is coded with fewer bits, and thus requires less time to be transmitted. 
This prevents buffer underflow at the receiving end. 

Optionally and preferably the transcoded data is also redimdancy encoded. 
So, if packets are dropped during the congestion, less information is not lost. 

The invention offers the advantage that network jitter is avoided. The user is 
not subjected to jitter, and does not miss out on the continuity of the streaming media. 

The invention will become more readily apparent from the following Detailed 
Description, which proceeds with reference to the drawings, in which: 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a server in the prior art transmitting data to a 
client over a network. 

FIG. 2 is a time diagram illustrating why in the arrangement of FIG. 1 there 
occurs the phenomenon of network jitter that disrupts play out. 

FIG. 3 is a block diagram of a server made according to the invention 
transmitting data over a network to a chent. 

FIG. 4 is a flowchart illustrating a method according to the invention. 
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5 FIG. 5 is a time diagram illustrating why the phenomenon of network jitter is 

avoided in the arrangement of FIG. 3. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S) 

As has been mentioned, the present invention provides a server for 
10 transmitting streaming media to the network, that accounts for bandwidth availability 
fluctuations. Various aspects and embodiments of the invention are now described in 
more detail. 

Referring to FIG. 3, a device according to the invention is a server 310, which 
is also known as a proxy 310. As shown, the server 310 can be either a full server, or 
15 a portion of a server. The server 3 10 is for transmitting data over a network 1 50, such 
as a packet network, to a client 360. The client 360, which is also known as a receiver 
360, includes at least one de-jitter buffer 364. Optionally it also includes a variable bit 
rate decoder 366. 

The server 310 preferably includes a transmit buffer 314, that is also knovm as 
20 transmitting means 314. Buffer 314 is for buffering data received from a source, and 
for transmitting to the network 150 at the regular rate, such as at a fixed frame rate r 
required by the streaming media in question. The frame rate r may be, for example, 
100 frames per second for all audio, or 25 frames per second for video. The streaming 
media may have a constant or a variable transmission rate. When transmitting canned 
25 media, the transmit buffer 3 1 4 is not necessary. 

The server 310 moreover includes means for monitoring a bandwidth of the 
network 150. This feature can be optionally implemented in conjunction with control 
unit 335. This feature is used to provide an estimate of how fast data is being 
received at the client 360. This in turn is used to predict how full is the receive buffer 
30 364. 

The server 310 also includes a transcoder 340, made as is known in the art, 
which is otherwise known as transcoding means. The transcoder 340 is for 
transcoding the buffered streaming media, which is output through the transmit buffer 
3 14. This is preferably enabled only when the monitored bandwidth of the network 
35 150 becomes less than a first preset bandwidth. Once transcoded, the same number of 
frames uses less bandwidth. 

The server 310 preferably also comprises a control unit 355. The control unit 
355 is primarily for controlling the transcoder 340. Confrolling is mainly by 
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5 activating the transcoder 340 when the monitored bandwidth is less than the first 

preset value. In addition, the transcoder 340 is deactivated when the bandwidth of the 
network is restored to its full bandwidth. The control unit 355 can be implemented as 
a processor, software in a processor, a separate circuit, etc. 

The server 310 preferably also includes a redundancy encoder 350, which is 
10 also knovra as redundancy encoding means 350. The redundancy encoder 350 is for 
redundancy encoding the transcoded data, if the monitored bandwidth is less than a 
second preset value. Preferably the control unit 355 activates the redundancy encoder 
350 when the monitored bandwidth is less than the second preset value. The second 
preset value may be the same as the first preset value, although that is not necessary. 
15 It is readily apparent that the present invention can be implemented in logic 

circuitry, in a dedicated microcontroller circuit, or in a general pvirpose computer 
adapted with software. The invention also provides software that can practice 
methods of the invention. 

Referring to FIG. 4, a method of the invention is described with reference to 
20 flowchart 400. The method is for a server to retransmit streaming media to a network. 
The streaming media is encoded in data, and ultimately destined for a client. 

According to an optional box 410, a next portion of the streaming media is 
received in a server. According to a box 420, the received portion is stored in a 
transmit buffer of the server. 
25 According to a box 430, the available bandwidth of transmission is monitored. 

This can be accomplished by monitoring the available bandwidth of the network. 

According to a box 440, it is inquired whether the monitored bandwidth is less 
than a first preset bandwidth. That is a measure of how congested is the network. 

If it is, then according to box 450, the next portion of the streaming media is 
30 transcoded. It can be transcoded by being played out of the transmit buffer into a 
transcoder. Alternately, the data that encodes the streaming media can always be 
routed through the transcoder, which in turn is activated when a certain network 
congestion is determined. 

If the congestion is no longer detected, transcoding can stop. Nevertheless, it 
35 is suggested to continue transcoding for a while. This will give the opportunity for 
the bottlenecked transcoded frames to be entered into the receiving buffer. 

Continuing for box 450, according to a box 460, it is inquired whether the 
monitored bandwidth is less than a second preset bandwidth. The second preset 
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5 bandwidth can be the same as the first preset bandwidth, in which case the inqmry of 
box 460 is resolved at box 440. 

If the monitored bandwidth is less than the second preset bandwidth then, 
according to a box 470, the next portion is redundancy encoded. This is accomplished 
by having the nth packet contain the nth frame, as well as the transcoded (n-m)th 
10 frame. Even if packets are dropped during network congestion, and less than m 
consecutive packets are dropped, the receiver can still decode the frame using the 
redundant coded data. 

Redundancy encoding can be by routing the data through a redimdancy 
encoder, and activating it when a certain network congestion is detected. Alternately, 
15 the data can be switched aroimd a different path withm the server. 

Execution then proceeds to box 480 where, whether transcoded or not, the 
next portion of the streaming media is transmitted to the network. 

The performance of the invention can be now appreciated with reference to 
FIG. 5. Various quantities are plotted against a time axis, which is shown as a line 
20 510. 

A solid, piece-wise Unear graph 220 represents the time evolution of the 
available bandwidth of network 150. It will be observed that graph 220 is identical in 
time evolution to graph 220 of FIG. 2, with wells A and B. The similarity is 
intentional, so that the performance of the system of the invention can be compared to 

25 that of the prior art, under the same conditions. 

A dashed line 530 represents the instantaneous rate of receiving data through 
the network 150 in client 360. In FIG. 2, the line 530 starts at a value fy, which is the 
steady state, assuming no prior interruptions in the transmission. The value fy is 
again the streaming media play out rate, which is how fast the data stream is being 

30 transmitted from the server 310. 

Line 530 starts out less than the mstantaneous value of graph 220. However, 
during the well A and well B, the monitoring means instructs the server 310 to 
activate the transcoder 340. The subsequently arriving data is transcoded. This 
means that the frames are each having less data, and are therefore being registered 

35 faster. So, when well A and well B start, line 530 dips below the line fy, but then 

compensates by going higher. After a long, iminterrupted regular receiving time, line 
530 settles back at a value fy. 
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5 A line 540 represents the instantaneous fullness of the buffer 364. In FIG. 5, 

initially the buffer 364 is at its nominal size fy, after a long time of network 
congestion. Again, line 540 is given by an integral of a time difference between line 
530 minus value fy. As such, it follows with two wells A", B" the wells A, B, 
respectively. In fact, when line 530 catches up with value fy, that is when wells A", 

10 B" bottom out (have a local minimum). 

Notice that well B" never reaches zero. The buffer 364 does not underflow. 
This means that network jitter is avoided, where it was not avoided in the prior art of 
FIG. 2. 

During the congestion periods of wells A and B, however, the received data is 

15 transcoded, not regular. This is between the pair of time points 542 and 544 for well 
A", and the pair of time points 546, 548 for well B". 

When the transcoded data is played out, there is a reduction in quality. 
Nevertheless, the user does not feel he is missing out on any information, and may 
tolerate the play out during the congested time. 

20 A person skilled in the art will be able to practice the present invention in view 

of the description present in this document, which is to be taken as a whole. 
Numerous details have been set forth in order to provide a more thorough 
understanding of the invention. In other instances, well-known features have not been 
described in detail in order not to obscure unnecessarily the invention. 

25 While the invention has been disclosed in its preferred form, the specific 

embodiments thereof as disclosed and illustrated herein are not to be considered in a 
limiting sense. Indeed, it should be readily apparent to those skilled in the art in view 
of the present description that the invention can be modified in numerous ways. The 
inventor regards the subject matter of the invention to include all combinations and 

30 subcombinations of the various elements, features, functions and/or properties 
disclosed herein. 

The following claims define certain combinations and subcombinations, which 
are regarded as patentable. Additional claims for other combinations and 
subcombinations of features, fiinctions, elements and/or properties may be presented 
35 in this or a related document. 
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5 

The invention claimed is: 



CLAIMS 



1 . A server for transmitting stored data to a network, comprising: 
a transmit buffer for transmitting the stored data to the network; 
10 a network bandwidth monitor for monitoring a bandwidth of the network; and 

a transcoder for transcoding the stored data if the monitored bandwidth is less 
than a first preset value. 



2. The server of claim 1, wherein the monitoring means includes 

1 5 a control unit for activating the transcoder when the monitored bandwidth is 

less than the first preset value. 

3. The server of claim 1, further comprising: 

a redundancy encoder for redundancy encoding the transcoded data if the 
20 monitored bandwidth is less than a second preset value. 



4. The server of claim 3, wherein the monitoring means includes 

a control unit for activating the redundancy encoder when the monitored 
bandwidth is less than the second preset value. 

5. The server of claim 4, wherein the first preset value equals the second preset 
value. 



6. A server for transmitting data to a network, comprising: 

30 transmitting means for transmitting the data to the network; 

monitoring means for monitoring a bandwidth of the network; 

transcoding means for transcoding the data if the monitored bandwidth is less 
than a first preset value; and 

redundancy encoding means for redundancy encoding the transcoded data 
35 prior to transmission if the monitored bandvddth is less than a second preset value. 

7. The server of claim 6, wherein the first preset value equals the second preset 
value. 
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8. The server of claim 6, further comprising: 

a control unit for activating the transcoding means when the monitored 
bandv^idth is less than the first preset value and for activating the redundancy 
encoding means when the monitored bandwidth is less than the second preset value. 



9. The server of claim 8, wherein the first preset value equals the second preset 
value. 



10. An article comprising: a storage medium, said storage medium having stored 
1 5 thereon instructions for a server to transmit a portion of streaming media to a network, 
that, when executed by a computing device, result in: 

transmitting the portion to the network; 

monitoring a bandwidth of the network; and 

transcoding the portion prior to transmitting if the monitored bandwidth is less 
20 than a first preset value. 



1 1 . The article of claim 10, further comprising: 

activating a transcoder when the monitored bandwidth is less than the first 
preset value. 

12. The article of claim 10, further comprising: 

redundancy encoding the transcoded portion if the monitored bandwidth is less 
than a second preset value. 



30 13. The article of claim 10, fiirther comprising: 

activating a redundancy encoder when the monitored bandwidth is less than 
the second preset value. 



14. A method for a server to transmit a portion of streaming media to a network 
35 comprising: 

transmitting the portion to the network; 
monitoring a bandwidth of the network; and 
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transcoding the portion prior to transmitting if the monitored bandwidth is less 
than a first preset value. 



1 5. The method of claim 14, fiirther comprising: 

activating a transcoder when the monitored bandwidth is less than the first 
10 preset value. 

16. The method of claim 14, further comprising: 

redundancy encoding the transcoded portion if the monitored bandwidth is less 
than a second preset value. 

15 

17. The method of claim 14, further comprising: 

activating a redimdancy encoder when the monitored bandwidth is less than 
the second preset value. 



PATENT APPLICATION 



10 



2705-125 



SERVER AND METHOD FOR TRANSMITTING STREAMING MEDIA 
TO CLIENT THROUGH A CONGESTED NETWORK 

ABSTRACT 

A server is provided with a transmit buffer, a transcoder, and a redundancy 
encoder. The server is for transmitting streaming media to a client through a 
communication network. A control unit senses when the network is temporarily 
congested, and activates the transcoder and the redimdancy encoder. The frames 
require fewer bits, and thus arrive in the client's de-jitter buffer without interruption, 
notwithstanding the network congestion. This prevents network jitter. 
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